Robust Meeting Event Recognition with a Multi-Modal Mixed-State Graphical Model

Meetings are social events, were people exchange information. Often a summarization of the meeting is necessary, for example for people not attending the meeting or to fix decisions. A first step for the automatic analysis of the meetings is a segmentation into meeting group action events like discussion or presentation [4]. This structuring can then be used to produce an agenda and a summarization of the meeting. Different approaches for this structuring, based on hidden Markov models (HMMs) [4] and dynamic Bayesian networks (DBNs) [5, 3] have been introduced for clear data sets. However, in real meetings the data can be disturbed in various ways: events like slamming of a door may mask the audio channel or background babble may appear; the visual channel can be (partly) masked by persons standing or walking in front of a camera. In this work we present a novel multi-modal mixed-state dynamic Bayesian network (DBN) [1, 2] for robust meeting event classification from disturbed videos.